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ABSTRACT 

In 1981 the Louisiana Board of Elementary and 
Secondary Education established regulations of quality assurance for 
educational program evaluations conducted in the state through two 
major areas: the cerjtif ication of educational program evaluators and 
the use of the Joint Committee's Standards for Evaluation, Projects 
and Materials in the evaluations themselves. This paper discusses the 
issues that must be faced in developing an application mechanism, and 
in articulating the relationship between the certification of 
evaluators and quality assurance of evaluations. The rolej9t the 
S^tate Department of Educatioii (SDE) and certified educators in the 
quality assurance ef lort in^ providing technical assistance and 
training in evaluation skills is also discussed. The role of the 
state in developing and disseminating models or guidance for local 
school systems is considered. The guidelines for implementing 
application of the Standards will be developed by the SDE Bureau of 
Evaluation in 1982. (PN) 
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APPLICATION OF THE JOINT COMMITTEE STANDARDS AS CRITERIA 
FOR EVALUATIONS IN LOUISIANA 

Introduction ' " ' 

In January of 1981 the Louisiana Board of Elementary and 
Secondary Education (BESE) adopted Rule 3.01 .70v(3ii) establishing 
the BESE's regulations of quality assurance for educational progran 
evaluations conducted In the state. The Rule approaches qOality 
assurance through two nJiajor areas: the certification of educational 
program evaluators and the use of adopted standards in the 
evaluations themselves* 

Three Items in these regulations address evaluation quality 
control. As approved by the BESE, these are: . 

1. The AERA Standards for Evaluation of Educational 
Programs, Projects and Materials should be used 
as the criteria for judging evaluations perforned 
for and by the state and local education agencies 
in Louisiana for project evaluators. 

2. The Standards should be monitored and 
Implemented through a "Standards Checklist" 
which should be field-tested by the Department of 
Education and % revised as needed by the Task 
Force on Program Evaluation. The "Standards 
Checklist" should be published fbr ready access 
to educators and evaluators In Louisiana. 

3. Training In the application of the AERA Standards 
In the use of the "Standards Checklist" should be 

a part of the required Inservice training program | 
prior to entry In the Louisiana Program Evaluator 
Registry. 

Guidelines for implementing the evaluator certification aspect of 
the regulation, adopted In June of 1981, led to some cfarlfication and 
minor changes In the Interpretation of the original Rule. The AERA 
Standards (referred to In the future as the BESE^-adopted Standards, 
or Standards) were In fact those developed by the Joint Committee on 
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Standards for Educational Evaluation and published as the Standards 
for Evaluations of Educational Programs, Projects and Materials 
(1981) , Thb Louisiana Program Evaluator Registry mentioned in item 
3 above will not be published as , originally titled* Certified 
evdiuators will instead be registered in the sense that their 
certification is issued by the State Department of Education (SDE) 
Bureau of Higher Education and Teacher Certification. The Bureau 
of Evaluation also maintains and distributes a Directory of persons 
who have completed certification training although this is not 
published as a listing of "state approved" evaluators. 

One final change from the original wording of the Rule is that 
the "Standards Checklist" described in Items 2 and 3 was not 
developed, field tested and reviewed by the Task Force on Program 
Evaluation* Prior to its dismissal, the Task Force on Program 
Evaluation, which had advised the BESE in forming Its regulations 
and guidelines, did not reach consensus about the content and 
structure of a checklist which could be used to judge evaluations on 
their adherence to the Standards. Those persons who participated in 
the original certification training were referred to an adaptation of 
the Citation Form wfcich comprises Appendix B of the Joint 
Committee's Standards . At this point the Bureau of Evaluation is 
working with other SDE staff and Louisiana evaluators to develop 
guidelines through which the BESE-adopted Standards will be applied 
to evaluations. This paper will discuss the issues that must be faced 
In developing an application mechanism which is practicable, 
professionally acceptable, and In keeping with the intent of Rule 
3.01.70v(3«). 



Objectives 

This paper has three purposes. One, it will articulate the 
relationship between the two major areas of the BESE Regulation, 
certification of evaluators and quality assurance of evaluations. Two, 
it wili discuss the possible role of the SDE and of certified evaluators 
in the quality assurance effort. Three, the paper will discuss issues 
in applying the Standards to educational evalu9tions in Louisiana. 
Evaluator certification and application of the Standards are interwoven 
as components in the total quality assurance effort, and personnel 
resources and audience needs will be as important in the application 
of the Standards as is the actual Regulation. The initial Task Force 
on Program evaluation, whose deliberations formed the basis of the 
BESE Regulations, included practicing evaluators from the SDE, local 
educational agencies, and the universities. The inplementation of 
quality assurance must remain a similarly cooperative effort if it is to 
meet the BESE intent of improving evaluation products and use. 

Relationship Between Certification and Quality Assurance 

In addition to the statements concerting application of the 
Standards, the BESE Regulation included the qualifications for 
certification as a program evaluator. Two levels of evaluator were 
defined. Level A evaluators were defined as those persons who 
designed, • approved, and directed educational evaluations; Level B 
evaluators, with less stringent courseworl< and experience 
requirements, were those who worked under the directlon^of a Level 
, A, The guidelines to the Regulation stipulated that educational 
evaluations conducted for or by the SDE should be carried out under 



the direction of a Level A evaluator. This Is the basic relationship 
between the certification and quality assurance aspects of the 

i 

Reguiations: evaluations received by the SDE must meet the criteria 
of the BESE-adopted Standards and must be conducted by persons 
certified as having been trained In the application of these Standards. 

At the recommendation of the SDE the six months following the 
adoption of Rule 3.au70v(34) were set aside as a "grandfatherinp" 
phase. During this time the Bureau of Evaluation offered training 
in the Standards to all persons referred by the State Superintendent 
of Education or local Superintendents whose written Job descriptions 
and current job responsibilities showed that they were functioning as 
Level A evaluators and that this was a. major part of their 
professional responsfbllitles. Completion of the training led to 
certification as a Level A evaluator. Coursework, degree and work 
experience requirements were waived. This Initial certification 
training and the characteristics of the evaluators certified during the 
grandfathering phase are discussed fully In another paper\ The 
participants In that training Represented the most visible, active 

educational evaluators In Louisiana. Virtually all had completed 

if 

graduate work beyond the baccalaureate, but their coursework 
Included little formaf training In evaluation. The majority had five or 
more years' experience as practicing evaluators and their job titles 



"Training In the Joint Conmlttee Standards: Content, Process, and 
Outcomes." J. Rachal^ Presented at AERA Annual Conference, New 
York, N.Y., 1982. 
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suggested that they worked primarily with federally funded catepor- 
ical programs. About two"*thirds saw themselves as independent or 
supervising eva^uators and comparable proportions reported that they 
were responsible for all evaluation tasks with the exception of those 
relating to contractual matters. They did not expect their evaluation 
respohsibllities to change In the Immediate future. These findings 
suggested two disparities: that between formal training and practical 
experience, and that between formula monitoring of categorical pro- 
grams and the comprehensive program evaluations reflected in the 
Standards. 

Role of the SDE a nd Certi fjed Evaluations in Quality Assurance 

These disparties point out in turn two major areas in which SDE 
staff and other certified evaluators will be active in the application of 
the Standards. They are particularly important if the nature of 
educational evaluation changed In Louisiana. The Bureau of 
Evaluation has taken tiie stance that there is a qualitative distinction 
between evaluations of programs which include specific evaluation 
guidelines, such as Title I, to be followed by the local education 
agency evaluator and those for which the evaluator must design and 
Implement an evaluation unique to a specific program or project. 
While both forms of evaluations will In all likelihood be required to 
demonstrate application of the Standards, the latter require consi- 
derably more sophisticated evaluatoi" skills. Very simple compliance 
monitoring, such as showing that a given number of eligible clients 
received prog ram--specl fled services In accord with fiscal stipulations, 
will probably not be included under the 6ESE Regulation. 



As noted earlier, the certification workshop participants did not 
expect their evaluation responsibilities to change in the immediate 
future. Other factors suggest that they may well do so, and this will 
be measured In a Bureau of Evaluation study . There are several 
Indicators that more evaluation information will be demanded at the 

state level: ^ 

- ' J \ 

• When the Bureau of Evaluation was established | 

two years ago It was given the responsibility of 
developing, with the Task Force on Program 
Evaluation, the BESE Regulations and Guidelines 
for quality assurance. Once these arc in effect, 
they are likely to be used. ^ 

• Both the BESE and the state legislature have 
demanded evaluations of state-funded programs 
and projects, something rarely required in the 
past. Some of these, such as a $3,000,000 
reading program, have required an evalua^n 
conducted at the state level. Others, such as^a 
cluster of pilot projects In compensatory 
education, have required both state and local 
evaluations. 

• Louisiana has instituted what will probably prove 
to be the most comprehensive minimum competency 
testing program In the United States. Beginning 

/ with the 1981-1982 school year, all second grade 
{ students must take a state Basic Skills Test and 
I subsequent g rade levels will be added to the ^ 
I testing program at a rate of one per year until all 
grade 2-12 public school children are tested 
annually. This testing program is accompanied 
by the requirement that all children who fail to 
meet the required minimum perfornance level on 
the test will be provided with compensatory 
education. As well as participating in the state 
evaluation of the Compensatory education program, 
all local school systems will be required to 
conduct local evaluations which address the 
Standards. 



"Meta-evaluatlon of Education Evaluation in Louisiana: The Impact of 
Quality Assurance," W. E. Schroyer. Presonted at AERA Annual 
Conference,. New York, N.Y., 1982. V 

ERjc ; » g. 



• Federal funding changes will in all probability 
lead to changes In federal eval uat ion requi re- 
ments. ^As federal dftllars flow through the SDE, 
the Bureau of Evaluation expects a greater 
demand on the part of the state legislature and 
the BESE for evaluation information from the local 
education agencies about program effectiveness. 

In short. It is expected that within the next two to three years 
local school system evaluation requirements will increase and will 
require greater sliills on the part of educational evaluators. Many of 
these will be the persons who are already certified with Level A 
status. 

The arguments above project that the state will become a greater 
consumer of evaluations in the future and that the SOE will play an 
i increased role In setting guidelines for, and derr.andlng, local 
cK^aluations. The complement to this role is providing support for 
local evaluators. Tabl6 1 shows the interest in future training among 
the 21^ Initial certification training participants. The great majority 
indicated at the close of the workshops that they would be interested 
In future voluntary workshops. The topics selected by the largest 
numbers were evaluation models (5^%) and evaluation design (581). 
Slightly fewer numbers wanted training in questionnaire construction, 
data gathering and manipulation, and surveys and samplTng. The 
most frequently requested subjects were those specific to evaluation 
and which reflect total evaluation development rather than following 
prescribed guidelines. 



TABLE 1, 



INTEREST IN FUTURE TRAINING, 214 PARTICIPANTS 
IN EVALUATOR CERTIFICATION TRAINING 



NUMBER PERCENT 

Interest In Future Voluntary Workshops 
in Evaluation Topics 

Ves 185 86% 

No ^ 9 ^ 

No Answer 20 9 

Topics of Interest 

Basic Statistics Review 79 37% 

intermediate Statistics Review 42 20 

Evaluation Models 116 54 

Evaluation Design 124 ^ 58 

Questlonaire Construction 112 52 

Data Cathering/Manipulatlofi lC4 49 

Surveys and Sampling 100 47 

Report Preparation/Presentation 109 34 

Other Topics 72 34 

No Answer * 12 6 
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Practicing evaluators were part of tlie Task Force on Program 
Evaluation and remained very active in the later certification training. 
From them, and from other professional networks which have arisen, 
has come a list of suggestions for providing technical assistance to 
practicing evaluators. 

• The SDE could provide technical assistance to all 
evaluations required to meet thrf BESE-adopted 
Standards. This will occur to a limited degree, 
and ^ is already a part of those evaluations 
conducted by the Bureau of Evaluation which 
require an additional local evaluation. Carried to 
extremes, however, this could result in the 
Bureau's acutaily conducting all of the evaluations 
in the state and would obviate the need for the 
BESE qu^ity assurance regulations. It could also 
lead to potential conflict of interest. 

• The SDE could provide training in i^aluation 
skills. This will also be done. The Bureau of 
Evaluation is planning to offer a scries of 
workshops in those evaluation topics requested 

• most frequently during the certification 
workshops. Again, staff resources limit the 
extent of this inservice training and other 
Bureaus In the SDE do not possess the staff to 
do this. The Bureau's Inservice training will be 
necessarily short and directed toward limited 
topics. It cannot address basic deficiencies or 
replace graduate level training. 

t Universities could provide graduate level 
preservlce and Inservice education In evaluation. 
This Is tfi|e best formal approach In the long run, 
but there imay be ^ gap of several years between 
the need for evaluation skills and the presence of 
sufficient persons who possess them. At least 
four universities have expressed an Interest in 
developing master's or doctoral programs in 
educational evaluation, ^ 

f Experienced evaluators could mentor those with 
less experience* This was suggested by several 
. of the evaluators Involved In the development of 
the Regulations and the certification trainjlhg. 
Some of the larger school systems have research 
and evaluation components with Experienced and 
trained staffs that smaller systems cannot afford; 
they are also able to assign staff to educational 
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evaluation rather than adding it as one more hat 
a sincfle central office employee must wear. This 
suggestion should be pursued because it is a way 
of providing immediate and salient assistance and 
because it develops evaluation as a profession in ^ 
Louisiana* The major constraint would be the 
amount of time that mentoring evaiuators could 
provide* It also presupposes coordination and 
the willingness oi^ local evaiuators to share 
problems with others. 

• Models or guidance could be developed iocaiiy and 
disseminated through the SDE. One mechcT^ism 
for this could be the Louisiana Round Table of 
Program Evaiuators. This is a group of local 
school system e>>aluators that has met informally 
for the past year and a half, and which recently 
prepared a short monograph on developing need^ 
assessments. , The Round Table is an excellent 
structure for Identifying local needs and offering 
* practical^ nonacademic suggestions. A second 
mechanism Is the SDE Bureau of Dissemination*s 
"Promising Programs and Practices" file. This 
could be used to categorize and distribute model 
evaluations conducted by local evaiuators and 
♦ would serve as an incentive as well as an 
example. 

Ail of these suggest ways to approach the training or Information 
needs of practicing evaiuators. They do not address the actual 
implementation of the BESE-^dopted Standards. 



Issues in impienenting the Standards 

The guidelines for implementing application of the StandSirds 
have not yet been developed. They will be produced in the same way 
that the original Rule and guidelines for certification were: in early 
spring of 1982, study groups of local evaiuators In the Mortiiern and 
Southern parts of the state will meet to discuss ideas suggested by 
the Bureau of Evaluation. The suggestions of these study groups 
v;lll be incorporated into the Bureau's plans and submitted to the 
BESE for Its review and ultimate approval. At thi^ point p^^g^^m 
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guidelines from the SDE fmost notably for the Compensatory Education 
Program) require local educatron agency plans to address the 
BESE-adopted Standards* By the time that focal systems begin 
implementing their evaluation plans the guidelines will have been 
written and distributed* Several issues (problems or simple 
bacl<ground conditions) have already been identified by the Bureau of 
Evaluation and will be reflected In the eventual implementation 
guidelines* 

*> ♦ , 

1. Can the Joiht Committee's Standards, adopted by 
the BESE, actually be imp.lemented as criteria for 
judging the quality of evaluations? 

During the developmental worl<shop In June, 1981, 
several authors of the Standards trained a group 
of Louisiana evaluators in the Standards. Both 
the workshop presenters and the participants had 
some fears that applying the Standards as criteria 
of quality went against the original intent of the 
Joint Committee's publication. The Standards 
were not written as a "cooi< bool<" and using them 
as such could damage the evaluation profession by 
turning the Joint Committee into something lil<e a ^ 
licensing board or implying its sanction of poor 
• ^ quality evaluations. The guidelines will address 
this potential problem in three ways. First, the 
Bureau's staff has been in contact with 
representatives of the Joint Committee since , the 
inception of the Louisiana quality assurance 
program and has thus maintained a professional 
linkage between Louisiana's actions and the larger 
professional field. Second, the guidelines will 
probably require that evaluation iDlans submitted 
to the SDE address each of the Standards, 
indicating which are to be applied and which 
appear to be of less importance or not applicable. 
Third, ^evaluations will also be judged holistically. 
The evaluation plans will. In effect, show which 
principles will be used in the evaluation and the 
overall judgment of the evaluation will determine • 
whether the application of these principles 
resulted in a valid and adequate evaluation of the 
program. The Bureau's staff has already 
analyzed major categories of evaluations received 
by the SDE to determine which Standards are 
most likely to be crucial in each. 
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Which evaluations will be subject to the 
Standards? 

The BESE Requlattom is general enoucih to require 
that every evaluation performed for or by the 
state be judqed by its application of the 
Standards, and the guidelines will narrow this 
definition to avoid a potential nightmare of cost, 
paperwork, and staff time. The qualification 
being considered by the Bureau is total program 
cost: limiting application of the Standards to 
programs funded at $50/000 or more. The 
rationale behind this is that a minimal evaluation 
of programs this large should be about 5% of the 
total cost, or $2,500. Adequate evaluations of 
smaller programs would require a disproportibriiate 
share of program funds and smaller evaluations 
would probably be unable to address the 
Standards adequately, as well as flooding the 
eventual judging group with paperworl<. 

1^ pr6gram cost is used to determine whether an 
evaluation must address the Standards there will 
still have to be exceptions. Both the legislature 
and the BESE must be allowed to * require 
evaluations of programs they fund or approve, 
and program staff within the SDE who manage 
local programs must be granted the same 
prerogative. The last case is most likely when 
large amounts of SDE funds are distributed among 
a number of local school systems such that each 
receives less than $50,000. These evaluations 
which are exceptions to the funding limitation 
cut-off will probably require tandem evaluations, 
with those at the local evaluation coordinated or 
supplemented through an SDE evaluation of the 
total program. 

Will there be a "cost backlash?" 

This paper has already argued that there will in 
the near . future be a . greater demand for 
evaluation Information from state gi^oups 
approving and disbursing educational program 
funds. Most members of these groups have not 
been familiarized with the Stanaards and do not 
generally deslgn&te the amount of evaluation 
funds in program allocation^. * They, and SDE 
program staff, are often surprised at the cost of 
an adequate evaluation. The application guide- 
lines' should Include some recognition that 
.evaluations cannot be done for free, but the 
Bureau of Evaluation will also have to dp a good 
bit of education among^ evaluation users. « Part of 
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the rationale behind the BESE-approved 
I grandfathering certification period was to ensure 
that local educational agencies, and SDE Bureaus 
had a supply of Level A evaluators^ and were not 
required to spend large sums on tlilrd-party 
evaluators. A secoiSd means of reducinei cost will 
be to allow local evaluations of SDE-managed 
programs to cite the SDE program director or 
staff as the supervising Level A evaluator. 

4« • What sanctions will be employed in applying the 
Standards? 

The 6ESE f^egulations state simply that the 
adopted Standards "should be used as the criteria 
for judging evaluations." The guidelines must go* 
beyond this and state that the Standards :must be 
applied, in an acceptable manner, and to an 
acceptable degree. The other side of that coin Is 
determing what will happen if they are not. The 
first step Is the program approval stage: local or 
state program plans can be required to include an 
evaluation plan which shows how the evaluation 
will address the Standards. These plans can be 
negotiated until an acceptable evaluation is 
proposed, and there are precedents for such 
negotiations in other state programs. The 
subsequent steps are In judging whether an 
evaluation was actually conducted according to 
plan, and whether these activities resulted In an 
adequate evaluation. There is no guarantee that 
a "yes" at each step promises a "yes" at the 
following one. The most likely sanctions are 
fiscal. Inadequate plans can lead to the refusal 
to fund until an acceptable plan has been 
submitted. Program funds (if they are not totally 
distributed up front, as is the common practice) 
can be withheld if evaluations are being conduct- 
ed out of compliance with the plan* Refunding 
can be denied If the final evaluation product is 
not {acceptable. These are possible sanctions 
which may not be acceptable to the BESE. They 
also require constant monitoring on the part of 
the program's funding or managing staff, close 
cooperation between program and evaluation 
personnel, and local school system staffs with the 
time and expertise to carry out good evaluations. 

Sanctions include rewards as well as punishments, 
and three cdme Immediately to mind. The first, 
and most difficulty Is to promote the usefulness of 
evaluation information. The most typical current 
evaluations are thos^*^ of federal categorical 
programs, such as TitljJ I, which provide little 
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useful data to local program staff. The manner 
In which the Standards are Implemented should 
attempt to .make evaluations as useful to those 
conducting them as is possible, by integrating 
local evaluations with planning needs, public 
^relations, efforts, ongoing program management 
requirements, or other areas. A second reward 
lies potentially in public \ recognition of out- 
standing programs and good evaluations. This 
could be coordinated with the SDE^s established 
dissemination activities. The third possible 
positive outcome is the development of a 
professional networt< of evaluators. The Louisiana 
Round Table of Evaluators has continued for a 
year and a half with no stated .puf^pose other than 
sharing professional* problems and interests. 
During that time the meeti|ngs have been devoted 
to specific topics reflecting members* work 
(evaluation use. Item analysis in testing, needs 
assessment) and to open discussion with SOE staff 
about state programs affecting the local education 
agencies. The great majority of Louisiana's 
school systems have one person responsible for 
evaluation In addition to other duties, and the 
Round Table members have said that they come to 
the meetings to talk to others who know what 
they do. Recognizing evaluation as a profession 
and providing professional conversation is not a 
v;eak Incentive. 



Should the implementation mechanism be model 
based or model-free? I 



The guidelines could allow the SDE to require 
that local evaluations adhere to one of a list of 
classic models but will in all probability not do 
so. Prescribing r|nodels would chill the 
development of local evaluation skills, remove the 
process of evaluation design development which is 
seen as crucial to Information usefulness, and 
reduce the SDE's role to a monitoring one. 
However, the simple act of writing guidelines 
which specify how the Standards are to be 
addressed, and what will be the criteria for 
acceptable evaluations, creates a de fac to model in 
Itself. Since the Standards were not developed to 
comprise an evaluation model there Is an inherent 
contradiction In the process. But without some 
fairly specific guidance the Implementation could 
become a frustrating experience for local 
evaluators seeking to satisfy a state agency which 
refuses to say precisely what It wants. 




What will be the accountability mechanism? 

The procedures for judging evaluation plans and 
products may not include the conceptual difficul- 
ties of other issues, but they could turn out to 
be unmanagably complex In their operation. In 
all likelihood the Bureau of Evaluation will be 
ultimately accountable for all 6valuations covered 
by the guidelines. Some state programs, such as 
Compensatory Education , will be evaluated 
directly by the Bureau, which will prepare and 
present the final evaluation report. In such 
programs the Bureau can (a) approve local or 
third-party evaluations (b) conduct an evaluation 
which Is coordinated with required local 
evaluations or (c) conduct an independent 
evaluation. In all of these cases the Bureau must 
coordinate Its requirements ancl activities with 
both SDE program management staff and with local 
educational agency staff. The second category of 
programs are those reflected in federal grants. In 
which an SDE staff typically signs off on received 
reports. In these cases the SDE program 
personnel can serve as the responsible Level A 
evaluator. However, the Bureau of Evaluation is 
charged with responsibility for the quality 
assurance program and must have some develop- 
mental, review, or monitoring function in those 
evaluations with which it fs not directly involved. 
Here the coordination may prove to be even more 
difficult; while control Is reduced by distance 
from the evaluation activities, responsibility Is 
not. 

Two options have been considered by the Bureau 
of Evaluation for applying the Standards to local 
evaluations. The first Is the Inclusion of a 
checklist that would enumerate all of the 
Standards, lnd|catfng which were addressed and 
which were not. The second Is the development 
of a general report format that Incorporates the 
Standards without specifying an evaluation model. 
Either of these could be reviewed In the same 
ways. At the local level, evaluatldns could be 
reviewed by a Level A evaluator from the same 
school system or from another school system. 
The SDE could also provide a technical assistance 
or monitoring team to review evaluations locally. 
The first set of possibilities would have the great 
advantage of encouraging local Involvement and 
legitimizing the professional role of local 
evaluators, but could not be a final step In the 
accountability process. The second set could 
conceivably take care of accountability, if the 
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SDE teams were periodically monitored, but would 
require adding to SDE staff. 

At the state level, evaluations could be reviewed 
by the recipient of the report. Review guidelines 
would iikely have to be developed by the Bureau 
of Evaluation, which would also need to supply 
technclal assistance to other SDE staff. A second 
possibility would be for the Bureau to review all 
evaluations. This would mean a considerable 

increase In staff size and would put the Bureau ^ 

in a watchdog role It may not wish to assume. It 

would also increase the distance between 

evaluation production and use by pro^r^m staff. 

A third potential method would be to have 

evaluations reviewed by a panel of recipients. 

This adds more program staff input and increases 

the chance that evaluations will be judged 

holisticaily rather than as compilance activities. 

It also increases the burden on program staff 

time, involvement of Bureau of Evaluation staff 

In such a review\|: panel would meet the 

accountability requirement, as would such 

involvement In the two final suggestions: review 

by a panel of local education agency evaluators or 

by a panel employed for this purpose. Using 

local evaluators would certainly contribute to the 

overail quality of evatuation In Louisiana, but 

could raise some problems In asl<ing school . 

sys|tems to provide staff release time or using |p 

non SDE judgments In deciding whether or not 

sanctions should be taken against a program. 

The last suggestion would be costly, and while It 

might encourage objectivity would not develop 

local commitment or skills. 



Conclusions 

Although these issues have been broken out in clusters to 
organize their discussion, they are certainly not independent. The 
consideration of sanctions will be a part of accountability, the 
determination of programs subject to evaluation quality assurance will 
be a key tomj^onent In deciding how the implementation mechanism will 
work, and the question of whether or not the Stand^ds can be 
applied underlies aii of the other decisions. And, while Louisiana is 
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unique in its evaluation quality assurance program, the problems it 
faces in implementing tiiis would be applicable to otiier states 
considering a similar program. The BESE-adopted Standards will be 

« 

applied to evaluations conducted in Louisiana. The guidelines under 
which this is done will include troth compliance and technical 
assistance components. They will also attempt to Implement quality 
assurance in a way which is professionally sound, not unduly 
burdensome for local school systems, and will improve the quality of 
evaluation processes and products in the state. 
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